Content providers such as Multi System Operators (MSOs) may employ optical character recognition (OCR) to test the performance of their content delivery networks (CDNs). For example, an MSO may employ an OCR test function to analyze the delivery of content to user devices such as client devices (e.g., display devices, computers, servers, smart phones, tablet computing devices, internet-enabled television sets). However, typical OCR test functions often produce unpredictable or inconsistent results when used in video or video on demand (VOD) test automation. In addition, typical OCR libraries and test functions may not support dynamic comparison of text and other data from multiple video frames. Furthermore, typical OCR test functions provide no mechanisms for controlling the execution time of test scripts.
One solution is to use indexed looping to run OCR multiple times in a loop (e.g., 5 times). However, this approach may result in an unacceptably low number of OCR comparisons producing expected results due to false matches and uncontrollable execution time. As a result, video test automation results may require human validation and, in some instances, video test automation efforts may become abandoned in favor of manual testing. This disclosure provides solutions for these and other issues.